Analysis of mean-square error and transient speed of the LMS Adaptive algorithm

For the least mean square (LMS) algorithm, we analyze the correlation matrix of the filter coefficient estimation error and the signal estimation error in the transient phase as well as in steady state. We establish the convergence of the second-order statistics as the number of iterations increases, and we derive the exact asymptotic expressions for the mean square errors. In particular, the result for the excess signal estimation error gives conditions under which the LMS algorithm outperforms the Wiener filter with the same number of taps. We also analyze a new measure of transient speed. We do not assume a linear regression model: the desired signal and the data process are allowed to be nonlinearly related. The data is assumed to be an instantaneous transformation of a stationary Markov process satisfying certain ergodic conditions.